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11/20/00 



In the United States Patent and Trademark Office 



Attorney Docket No: Z032-C 



Anticipated Classification 
of this application: 

Class: Subclass: 

Prior application: 09/283,238 

Group Art Unit: 2 712 



Honorable Commissioner of 

Patent and Trademarks 
Washington, D. C. 20231 



Request for Filing a Patent Application Under 37 CFR 1.53(b) 
to Correct Inventorship where at least one of the Correct 
Inventors was named in the Prior Application 



This is a request for filing a 
[X] continuation 
[ ] divisional 

application under 37 CFR 1.53(b) of pending prior application serial 
no. 09/283 , 238 , filed on April 1, 1999 , entitled Instrument and Method 
for Digital Image Stabilization . 

1. [X] Enclosed is a copy of the prior application as originally 
filed and a new declaration executed by the two correct 
inventors . 



2. [ ] A verified statement to establish small entity status 

under 37 CFR 1.9 and 1.27 [ ] is enclosed [ ] was filed in 
the prior application and such status is still proper and 
desired [37 CFR 1.28(a)]. 
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3. [X] The filing fee is calculated below: 

Claims as filed in the prior application less any claims canceled 
by amendment below. 

Fee No. filed No. extra Rate Fee 

Basic fee $710.00 

Total claims 14 -20 = 0 x 18.00 $ 0 

Independent 2 -3 = 0 x 80.00 $ 0 

claims 

Multiple Dependent 

Claim presented 0 + 270.00 $ 0 

Total $710.00 



[X] The Commissioner is hereby authorized to charge payment of 
the following fees which may be required during the entire 
pendency of the application to Deposit Account No. 15-0773 
or credit any overpayment to said account : 

[X] any patent application filing fees required 
under 3 7 CFR 1.16; 

[X] any filing fees required under 37 CFR 1.16 for the 
presentation of extra claims; and, 

[X] any patent application processing fees required 
under 3 7 CFR 1.17. 



[X] A check in the amount of $710 . 00 is enclosed. 



[X] The inventors being claimed in the application are: 
Michael Claus and Karl -Heinz ESwein . 



[X] Amend the specification by inserting before the first line 
the sentence: -- This is a continuation of application 
serial no. 09/283,238, filed April 1, 1999. -- . 



[X] Please abandon the prior application as of the filing date 

accorded this application. A duplicate copy of this request 
is enclosed for filing in the prior application file. 
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8. [X] New drawings are enclosed. 

[X] formal drawings of FIGS . 1 and 2 



[X] Priority of application serial no. 198 14 951.4 filed 
on April 3, 1998 in the Federal Republic of Germany is 
claimed under 3 5 USC 119. 

[X] The certified copy has been filed in prior application 
serial no. 09/283 , 238 , filed on April 1, 1999 . 



10. [X] A new assignment is enclosed duly executed by the correct 
inventors . 



The power of attorney is set forth in the new declaration 
executed by the two correct inventors and is to : 

Walter Ottesen 
Registration Number 25,544 

[X] The power also appears in the original papers in the 
prior application. 

[ ] Since the power does not appear in the original 

papers, a copy of the power in the prior application 
is enclosed. 

[X] Address all future communications to: 

Walter Ottesen 
Patent Attorney 
P.O. Box 4026 

Gaithersburg, Maryland 20885-4026 



[X] An information disclosure statement is enclosed. 



[X] I hereby verify that the attached papers (specification and 
drawings) are a true copy of those of the prior application 
serial no. 09/283,238, as filed on April 1, 1999. 



The undersigned declares further that all statements made herein 
of his own knowledge are true and that all statements made on 
information and belief are believed to be true; and, further that these 
statements were made with the knowledge that willful false statements 
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and the like so made are punishable by fine or imprisonment, or both, 
under Section 1001 of Title 18 of the United States Code and that such 
willful false statements may jeopardize the validity of the application 
or any patent issuing thereon. 



Signatory is the Attorney of Record having Registration 
Number 25, 544 . 



Walter Ottesen 
Patent Attorney 
P.O. Box 4026 

Gaithersburg, Maryland 20885-4026 

Telephone: (301) 869-8950 
Telefax: (301) 869-8929 





(date) 



Walter Ottesen 
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Attorney Docket No: Z032 

Instrument and Method for Digital Image Stabilization 
Field of the Invention 

The invention relates to an arrangement and a method for 
digitally stabilizing image recordings with a CCD sensor mounted 
in a moving or flying carrier. 
Background of the Invention 

Stabilization systems known up to now operate in accordance 
with the electromechanical principle with a cardanic suspension 
of the sensor while driving the positioning motors via an 
inertial sensor; or, CCD sensors are utilized when making aerial 
image recordations without stabilization. Here, a migration of 
the images must be accepted for a line sensor between the lines 
or between frames with an area sensor. This image migration is 
caused by disturbing movements of the carrier. 

Generally, it is possible to provide image stabilization on 
board or in a ground station. In the first case, there is an 
on-line stabilization for display on a monitor and, in the second 
case, there is an off-line image evaluation. As a rule, the 
on-line stabilization is carried out. This takes place with area 
sensors in camcorders. An on-line correction is carried out and 
analog signal processing takes place. 

For example, European patent publication 0,543,394 discloses 
that, for an area sensor, the movements occurring at the time 
point of the exposure are corrected on-line in a frequency range 
of 1 to 12 Hz. The correction takes place via an optical 
element. The correcting signals are determined via two sensors 
which each measure the angular acceleration or the angular 
velocity. 

Furthermore, European patent publication 0,574,228 discloses 



the determination of vibrations in the horizontal and vertical 
directions. Here, an on-line correction is made either via 
readdressing or reading out from a storage register at a lower 
frequency than for writing. 

Finally, United States Patent 4,959,725 discloses a very 
complex circuit for correcting the video signal of an area sensor 
while performing an evaluation with acceleroraeters . 

However, all known solutions of the state of the art cannot 
be characterized as optimal with respect to their commercial 
effectiveness because of the high cost associated therewith. 
Summary of the Invention 

It is an object of the invention to provide an arrangement 
and a method for digital image stabilization with which the 
movement influences of a moving or flying carrier on the image 
quality of a sensor are compensated at low cost. The sensor 
moves or flies with the carrier. More specifically, a correction 
of the effects of roll and pitch of the aerial image carriers 
having line sensors and area sensors is made possible with at 
least pixel precision. This correction takes place digitally 
with large data quantities of > 6,000 pixels per line and 
therefore takes place off-line and is economical. 

The method of the invention is for digitally stabilizing an 
image recording with a CCD sensor, which is mounted in a moving 
or airborne carrier, for substantially eliminating unwanted 
movement influences of flight movements of the carrier on the 
image quality of the image recorded by the CCD sensor. The 
method includes the steps of: detecting an image with the CCD 
sensor and outputting image data; detecting the flight movements 
of the carrier as angular data with an inertial sensor and the 
inertial sensor being adapted to supply the angular data with a 



time delay; and, correcting the image data in accordance with the 
detected angular data with the image data being time delayed by a 
time interval relative to the detected angular data. 
Brief Description of the Drawings 

The invention will now be described with reference to the 
drawings wherein: 

FIG. 1 is a block circuit diagram of an arrangement of the 
invention for an on-line stabilization of image recordings; and, 

FIG. 2 is a block circuit diagram of an arrangement for an 
off-line stabilization of the image recordings. 
Description of the Preferred Embodiments of the Invention 

With the method and arrangement for digitally stabilizing an 
image in accordance with the invention, image stabilization can 
take place on board, that is, an on-line stabilization for 
display on a monitor or in a fixed ground station, that is, an 
off-line image evaluation. 

FIG. 1 shows a block circuit diagram of an on-line 
stabilization according to the invention and FIG. 2 shows a block 
circuit diagram of an off-line stabilization according to the 
invention. Here, the movements of the carrier during flight are 
detected as angle data 11 with the aid of an inertial sensor 2 
and are, together with the image data 10, intermediately stored 
or recorded on a data carrier 3 or 6 . 

For an image data correction, image lines are displaced by 
corresponding angular increments transversely to the direction of 
flight for stabilization about the roll axis; whereas, a 
stabilization about the pitch axis takes place via an omission or 
a duplication of whole lines. The image data of neighboring 
pixels are interpolated in and transversely to the flight 
direction for a stabilization accuracy in the subpixel range. 



The inertial sensor 2 detects the flying movements and can 
be a so-called "strap-down sensor" . For this sensor, the 
stabilization result is essentially dependent upon the scale 
accuracy and the bandwidth of the sensor. The inertial sensor 2 
has a finite bandwidth which is typically 100 Hz and therefore 
supplies the detected angular data 11 with a time delay. For 
this reason, its transmitting performance at low frequencies can 
be approached by a dead time element (linear phase, that is, 
phase lag proportional to frequency) . 

A dynamically calibrated gyro (DAK) is very well suited as 
an inertial sensor 2. With such devices, scale errors to 0.1% 
and bandwidths up to 100 Hz are achievable. 

For an exemplary embodiment of a dynamically calibrated 
gyro (DAK), which is used as an inertial sensor 2, the pixel size 
of the sensor (dx) was 12 jum, the bandwidth of the objective (f) 
was 60 mm; the angular expansion of the sensor (DAS = dx/f) was 
0.2 mrad and the line frequency was minimally 200 Hz and 
maximally 2,500 Hz. Sinusoidally-shaped disturbance excitations 
in the region of 10°/0.2 Hz to 0.2°/10 Hz have rotational 
increments of 12.5°/sec = 220 mrad/sec. 

For an exposure time of 0.4 ms, an image blurring 
of 0.0 9 mrad results which corresponds to 0.5 pixels; whereas, 
for an exposure time of 5 ms, an image blurring of 1.1 mrad 
results which corresponds to 5.5 pixels. Accordingly, an image 
blurring over several pixels must be accepted at low line 
frequencies which can no longer be corrected by the subsequent 
stabilization. 

A total error or the quadratic sum of the residual error of 
the digital stabilization and the image blurring amounts 
to 0.22 mrad in dependence upon the line frequency or line 



rate (LR) for LR = 2,500 Hz and to 1.12 mrad for LR = 200 Hz. In 
this way, the following results for the stabilization factor: 

Stabilization Factor for 
Disturbance Excitation: LR = 2,500 Hz (200 Hz) 

10°/0.2 Hz 792 (155) 

0.2°/l0 Hz 16 (3) 

The computer simulation of a dynamically calibrated 
gyro (DAK) having a bandwidth of 100 Hz and a scale error of 0.1% 
has shown that the stabilization error of the corrected image 
is <0.2 mrad (zero peak) for sinusoidally-shaped disturbance 
excitations in the range from 10°/0.2 Hz to 0.2°/10 Hz. 

A digital image stabilization is therefore purposeful when 
the frequencies of the disturbance excitation are so low that no 
intense "blurring" occurs during the exposure time. The use of 
stabilization can be evaluated by the stabilization factor which 
is the amplitude ratio of disturbance excitation and residual 
image movement . 

Especially even non-stabilized sensors can be economically 
expanded with an arrangement for digital image stabilization via 
corresponding modules and therefore made more valuable . 

A digital image stabilization according to the invention can 
take place as an on-line stabilization (FIG. 1) for display on a 
monitor 5 or as an off-line stabilization in a fixed ground 
station. In the on-line stabilization, an inertial sensor 2 is 
provided on board and a device for intermediate storage 3 of 
image data as well as a device for image data correction 4 . In 
the off-line stabilization, an inertial sensor 2 and an 
additional recording device 6 for angular sensor signals (angular 
data) are provided on board and, in the ground station, a 
device 4' is provided for carrying out an image correction in 



accordance with registered or recorded angular sensor signals 
(angle data) . 

The arrangement and method for digital image stabilization 
"use especially the concept of storing or time delaying the image 
data. For image correction, the image data are delayed relative 
to the recorded angle data by a time delay interval which 
considers the time delay caused by the inertial sensor as well as 
by scanning and computer time. 

For the on-line stabilization (on board the carrier) as 
shown in FIG. 1, a detection of image data 10 takes place via a 
CCD sensor. The CCD sensor is utilized in a moving or flying 
carrier. These image data 10 are intermediately stored in a 
device 3 and are outputted as time-delayed image data 10 ' to a 
device for image correction 4. Additionally, the device for 
image data correction 4 receives data from an inertial sensor 2 
(angle data) which reflect the movements of the carrier. The 
device for image data correction 4 generates a stabilized 
image 13 from the delayed image data 10 ' and the angle data 11 
and this stabilized image is displayed on a monitor 5 which 
likewise is disposed on board the carrier. 

In the off-line stabilization as shown in FIG. 2, a 
detection of the image data 10 takes place via a CCD sensor 2 
provided on board a moving or flying carrier. Furthermore, an 
inertial sensor 2 is configured on board the carrier which 
detects the flying movements of the carrier and outputs 
corresponding angular data 11. These angle data 11, together 
with the image data 10, are recorded in a recording device 6. 
For off-line evaluation and image data correction, the recorded 
image data and angle data 12 are outputted to a device for image 
data correction 4' in a ground station. This device for image 



data correction 4' delays the image data relative to the recorded 
angle data by a time interval which considers the time delay 
caused by the inertial sensor as well as by scanning and 
computation time . 
5 With the measures explained above, the problematic of 

unwanted movement effects of a moving or flying carrier on the 
image quality of a sensor (which moves with or flies with the 
carrier) is solved in a very economical manner via digital 
stabilization. 

10 A performance increase is achieved by an integration of the 

arrangement and method according to the invention into 

electrooptical sensor systems. 

It is understood that the foregoing description is that of 

the preferred embodiments of the invention and that various 
15 changes and modifications may be made thereto without departing 

from the spirit and scope of the invention as defined in the 

appended claims . 
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What is claimed is : 



1. A method for digitally stabilizing an image recording with a 
CCD sensor, which is mounted in a moving or airborne carrier, for 
substantially eliminating unwanted movement influences of flight 
movements of said carrier on the image quality of the image 
recorded by said CCD sensor, the method comprising the steps of: 

detecting an image with said CCD sensor and outputting image 

data; 

detecting said flight movements of said carrier as angular 
data with an inertial sensor and said inertial sensor being 
adapted to supply said angular data with a time delay; and, 

correcting said image data in accordance with said detected 
angular data with said image data being time delayed by a time 
interval relative to said detected angular data. 

2. The method of claim 1, comprising the further step of 
intermediately storing said image data from said CCD sensor on a 
data carrier to delay said image data by a time interval . 

3. The method of claim 2, wherein the correction of said image 
data is carried out as an on-line stabilization by performing the 
further step of displaying the corrected image data as a 
stabilized image on a monitor on board said carrier. 

4. The method of claim 1, comprising the further step of: 

recording the detected angular data together with the image 
data from said CCD sensor on a data carrier; and, 

carrying out the correction of image data as an off-line 
evaluation in a fixed ground station. 



5. The method of claim 1, wherein said time interval for 
delaying considers the time delay by said inertial sensor as well 
as by scanning and computation time. 

6. The method of claim 1, wherein said inertial sensor is a 
strap-down sensor. 

7. The method of claim 1, comprising the further step of 
shifting the image lines by pixels by the corresponding angular 
increments transversely to the direction of flight when 
correcting said image data to stabilize about the roll axis while 

5 stabilizing about the pitch axis takes place by omitting or 

reproducing whole lines. 

8. An arrangement for digitally stabilizing an image recording 
with a CCD sensor, which is mounted in a moving or airborne 
carrier, for substantially eliminating unwanted movement 
influences of flight movements of said carrier on the image 

5 quality of the image recorded by said CCD sensor, the arrangement 

comprising : 

said CCD sensor being provided to detect images and to 
output image data; 

an inertial sensor for detecting movements of said carrier 
10 as angular data; 

a first device for correcting said image data in accordance 
with said angular data; and, 

a second device for delaying said image data by a time 
interval relative to said image data. 
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9. The arrangement of claim 8, wherein said second device is an 
intermediate store for delaying said image data by said time 
interval . 

10. The arrangement of claim 8, wherein said first device is 
mounted on board said carrier and is adapted to carry out an 
on-line stabilization; and, wherein said arrangement further 
comprises a monitor on board said carrier for displaying the 
corrected image data as a stabilized image. 

11. The arrangement of claim 8, further comprising: 

a unit for recording said angular data from said inert ial 
sensor together with said image data on board said carrier; and, 

said first device for correcting said image data being 
mounted in a fixed ground station and said first device being 
adapted to carry out an off-line image evaluation. 

12. The arrangement of claim 8, wherein the time interval for 
delaying considers the time delay by said inertial sensor as well 
as by scanning and computation time . 

13. The arrangement of claim 8, wherein said inertial sensor is 
a strap-down sensor. 

14. The arrangement of claim 8, wherein said first device is 
adapted to shift the image lines by pixels by the corresponding 
angular increments transversely to the direction of flight when 
correcting said image data to stabilize about the roll axis while 
stabilizing about the pitch axis takes place by omitting or 
reproducing whole lines . 



Abstract of the Disclosure 

The invention relates to a high-precision digital image 
stabilization of an image recording with a CCD sensor utilized in 
a moving or flying carrier. For the digital image stabilization, 
5 the movements of the carrier are detected with the aid of an 

inertial sensor and stored. Then, a correction of the image data 
is carried out based on the detected movements. 
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Attorney Docket No. Z032-C 
Declaration and Power of Attorney for Patent Application 



As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name. 

I believe I am the original, first and sole inventor (if only one name is listed below) 
or an original, first and joint inventor (if plural names are listed below) of the 
subject matter which is claimed and for which a patent is sought on the invention 
entitled: Instrument and Method for Digital Image Stabilization , the 
specification of which is attached hereto. 

I hereby state that I have reviewed and understand the contents of the above-identified 
specification, including the claims. 

I acknowledge the duty to disclose Information which is material to the examination of 
this application in accordance with Title 37, Code of Federal Regulations, §1. 56(a). 

I hereby claim foreign priority benefits under Title 35, United States Code, §119, of 
any foreign application (s) for patent or inventor's certificate listed below and have 
also identified below any foreign application for patent or inventor's certificate 
having a filing date before that of the application on which priority is claimed: 

Prior Foreign Application (s) Priority Claimed 

198 14 951.4 Federal Republic of Germany 03 Apr 98 X 

(Number) (Country) Date Filed Yes No 



(Number) (Country) Date Filed Yes t 

I hereby claim the benefit under Title 35, United States Code, §120 of the United 
States application listed below and, insofar as the subject matter of each of the 
claims of this application is not disclosed in the prior United States application in 
the manner provided by the first paragraph of Title 35, United States Code, §112, I 
acknowledge the duty to disclose material information as defined in Title 37, Code of 
Federal Regulations, §1.56 (a) which occurred between the filing date of the prior 
application and the national filing date of this application: 

09/283,238 01 Apr 1999 Pending 

Application Serial No. Filing Date Status 

As a named inventor, I hereby appoint the following attorney to prosecute this 
application and to transact all business in the Patent and Trademark Office connected 
therewith: 



Walter Ottesen 
Reg. No. 25,544 

Direct all telephone calls to Walter Ottesen at telephone no. (301) 869-8950 and 
address all correspondence to: 

Walter Ottesen 
Patent Attorney 
P.O. Box 4 02 6 

Gaithersburg, Maryland 20885-4026 

I hereby declare that all statements made herein of my own knowledge are true and that 
all statements made on information and belief are believed to be true; and further that 
these statements were made with the knowledge that willful false statements and the 



like so made are punishable by fine or imprisonment, or both, under Section 1001 of 
Title 18 of the United States Code and that such willful false statements may 
jeopardize the validity of the application or any patent issued thereon. 

Full name of sole or first inventor Michael Claus " 

Inventor's signature .■ L-wC-t { L^^^J Date /'^^^ ' y'^ ^ -^"c?~> . 

Residence 73434 Aalen, Federal Republic of Germany 

Country of Citizenship Federal Republic of Germany 

Post Office Address HermelinstraSe 30/l, 73434 Aalen 

Federal Republic of Germany 



Full name of second : 



Inventor ' 
Residence_ 



Date 



Country of Citizenship Federal Republic of Germany 
Post Office Address Hohe KreuzstraSe 58, 73434 Aalen 



Federal Republic of Germany 



